Poisson Latent Feature Calculus for Generalized Indian Buffet Processes

The purpose of this work is to describe a unified, and indeed simple, mechanism for non-parametric Bayesian analysis, construction and generative sampling of a large class of latent feature models which one can describe as generalized notions of Indian Buffet Processes(IBP). This is done via the Poisson Process Calculus as it now relates to latent feature models. The IBP was ingeniously devised by Griffiths and Ghahramani in (2005) and its generative scheme is cast in terms of customers entering sequentially an Indian Buffet restaurant and selecting previously sampled dishes as well as new dishes. In this metaphor dishes corresponds to latent features, attributes, preferences shared by individuals. The IBP, and its generalizations, represent an exciting class of models well suited to handle high dimensional statistical problems now common in this information age. The IBP is based on the usage of conditionally independent Bernoulli random variables, coupled with completely random measures acting as Bayesian priors, that are used to create sparse binary matrices. This Bayesian non-parametric view was a key insight due to Thibaux and Jordan (2007). One way to think of generalizations is to to use more general random variables. Of note in the current literature are models employing Poisson and Negative-Binomial random variables. However, unlike their closely related counterparts, generalized Chinese restaurant processes, the ability to analyze IBP models in a systematic and general manner is not yet available. The limitations are both in terms of knowledge about the effects of different priors and in terms of models based on a wider choice of random variables. This work will not only provide a thorough description of the properties of existing models but also provide a simple template to devise and analyze new models.Comment: This version provides more details for the multivariate extensions in section 5. We highlight the case of a simple multinomial distribution and showcase a multivariate Levy process prior we call a stable-Beta Dirichlet process. Section 4.1.1 expande

Poisson calculus for spatial neutral to the right processes

Neutral to the right (NTR) processes were introduced by Doksum in 1974 as Bayesian priors on the class of distributions on the real line. Since that time there have been numerous applications to models that arise in survival analysis subject to possible right censoring. However, unlike the Dirichlet process, the larger class of NTR processes has not been used in a wider range of more complex statistical applications. Here, to circumvent some of these limitations, we describe a natural extension of NTR processes to arbitrary Polish spaces, which we call spatial neutral to the right processes. Our construction also leads to a new rich class of random probability measures, which we call NTR species sampling models. We show that this class contains the important two parameter extension of the Dirichlet process. We provide a posterior analysis, which yields tractable NTR analogues of the Blackwell--MacQueen distribution. Our analysis turns out to be closely related to the study of regenerative composition structures. A new computational scheme, which is an ordered variant of the general Chinese restaurant processes, is developed. This can be used to approximate complex posterior quantities. We also discuss some relationships to results that appear outside of Bayesian nonparametrics.Comment: Published at http://dx.doi.org/10.1214/009053605000000732 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org